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Telepharmacy is an online application used to provide pharmaceutical 
services in the modern era. Therefore, this study aims to develop a 
questionnaire on a research model that integrates three well-known theories, 
namely the unified theory of acceptance and use of technology (UTAUT), 
the protection motivation theory (PMT), and the DeLone and McLean 
information success model, to determine factors that influence people’s 
behavior towards the adoption of telepharmacy services. A cross-sectional 
observational method was used with the purposive sampling technique. A 
total of 118 participants who were at least 18 years old received the 
questionnaire. They were selected using convenience sampling from a 
variety of population groupings. Data analysis was then performed with the 
exploratory factor analysis (EFA). Factor extraction was carried out using 
principal component analysis and varimax rotation. The results show 8 
factors consisting of social influence and support 15.54%; understanding of 
proficiency, confidence and quality of information 11.06%; condition of 
facilities and skills 6.64%; severity and suffering 7.87%; time opportunity 
support and system quality 7.74%; intention to adopt telepharmacy 7.34%; 
perceived vulnerability 6.80%; and attitude 6.37%. Furthermore, the overall 
construct formed can explain 72.40% of the total variance. These results 
indicate that the telepharmacy service adoption behavior questionnaire is 
valid and reliable. 
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1. INTRODUCTION 

COVID-19 has been declared a global pandemic by the World Health Organization (WHO) in 2020, 
and it has rapidly spread to several countries. On March 28", 2020, it was included in the “Very High” category 
based on the WHO risk assessment with a total of 571,678 cases and 26,494 death. Furthermore, these numbers 
have increased to 693,224 cases and 33,106 death as of March 30th, 2020. Reports also revealed that Europe 
and North America had become the epicenter of the COVID-19 pandemic, where the infection and mortality 
rates are higher compared to China. The United States has the highest number of positive patients with an 
average of 19,332 new cases. COVID-19 was first reported in Indonesia on March 2020 with two infected 
patients [1]. As of March 31*, the virus had infected 1,528 people with 136 deaths [2]. The COVID-19 mortality 
rate in Indonesia was 8.9%, which is the highest in Southeast Asia [1]. 
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In early March 2021, the government implemented a new policy to limit the level of direct interaction 
[3]. Telemedicine based on information and communication technology (ICT) is not limited to only when 
patients are in the hospital. A previous study revealed that the scope of ICT in telemedicine increased along 
with the level of acceptance and supporting tools. This technology provides an opportunity in supporting health 
care efforts for patients [4]. According to the Indonesian Ministry of Health 2016, some of the fields of 
telemedicine include teleradiology, telecardiology, and telepharmacy. One of the strategies that can be used 
during the pandemic is the implementation of telemedicine health services, which is beneficial to patients and 
doctors [5], [6]. Furthermore, COVID-19 is a deadly and contagious disease, hence, preventive measures, such 
as human distancing can save patients. Previous studies revealed that healthy people were affected during 
hospital visits [7]. A report released by [8] also showed that doctors and paramedical staff were exposed to 
COVID-19 with several death cases due to the infectious nature of the disease. Therefore, an alternative 
healthcare system in the form of telemedicine is needed to overcome these problems. In Africa, health systems 
based on telemedicine were successful in providing, reporting, and tracking patient healthcare services for 
Malaria and Ebola [9]. The WHO also confirmed that the use of e-health applications is efficient and cost- 
effective for surveillance [10]. 

Online application-based services have great potential to be implemented. The pharmaceutical 
industry can also implement COVID-19 health protocol as well as seek improvement in the business aspect 
using telepharmacy service. However, the biggest challenge is the adoption of these applications and websites. 
Telepharmacy is described as “the delivery of pharmacist care by licensed pharmacists and pharmacies to 
patients at a distant location through telecommunications” [11]. Medication selection, order review and 
dispensing, patient counseling and monitoring, as well as clinical service delivery are some of the services that 
have been integrated in the technology [11], [12]. By establishing remote dispensing stations, it can be 
successfully utilized in community pharmacy settings. The growing attention from the state government 
through regulations demonstrates the popularity and growth of telepharmacy [6]. 

Several theories, such as protection motivation theory (PMT) and theory of planned behavior (TPB) 
have been used to explain the influenza vaccine [13] as well as nation intention, and behaviors. Factors 
influencing the acceptance of work-from-home (WFH) technologies during the COVID-19 pandemic were 
studied by [14] using the extended unified theory of acceptance and use of technology (UTAUT) model and 
employing environmental concern with a 57.4% acceptance level. The DeLone and McLean IS Success Model 
is often used for IS research papers as a valid model and framework for complex dependent variables [15]. 
Furthermore, this model was developed and proposed to information systems to study user behavior as well as 
provide an explanation through the use of the technology. Several components can influence adoption behavior 
of technologies [16], [17]. This survey tool has also been administered to European and American populations, 
where different socio-demographic and cultural factors influenced adoption behaviour. The existing 
questionnaire shows an inadequate validity and reliability profile. Some of the limitations of previous 
measurement instruments were addressed by providing a brief, contemporary, comprehensive, valid, and 
reliable self-report assessment. Therefore, this study aims to develop a questionnaire on a research model that 
integrates three established theories, namely the UTAUT, the PMT, and DeLone and McLean information 
success model, to determine factors influencing people’s behavior towards the adoption of telepharmacy 
services. 


2. RESEARCH METHOD 
2.1. Study design and tools 

A mixed study methodology was used to construct and validate the questionnaire. Phase 1 consisted 
of a literature review and qualitative study, which was carried out to identify the constructs and items. Phase 2 
involves a quantitative study, which was performed to validate the questionnaire. Ethical permission was 
granted by the Ethics Commission of the Faculty of Medicine, Universitas Islam Sultan Agung with reference 
number 201/V1/2022/Commission on Bioethics. The study participants were asked to fill an informed consent 
online through a google form in Central Java, Indonesia. 


2.2. Survey development 

The development of the questionnaire was carried out using a methodological process consisting of 
four basic steps, namely literature review, focus group discussions, expert evaluation, and pilot testing [18]. 
Furthermore, a thorough literature review was performed as the initial phase to generate items for electronic 
search engines. A total of 13 factors and 44 items were identified based on previous studies Seethamraju et al. 
[19] dan Gao et al. [20], [21] Sun [22]; Bossen [23], [24]. One specialist from the field of linguistics, five 
experts from pharmaceutical academics, and information technology experts, examined the questionnaire 
developed to assess its validity, content, and critical appraisal. Data face validity from 10 respondents was 
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collected in June 2022 through a web-based questionnaire using google forms. The data obtained was used to 
measure the simplicity, clarity, overlapping responses, balance, use of jargon, the accuracy of responses, use 
of technical language, application, and relationship to problems. Final field study, information was collected 
with a web-based survey using a google form in July 2022. A total of 118 respondents who were at least 18 
years old filled the questionnaire. The participants were selected using convenience sampling from a variety of 
population groupings to achieve the greatest possible diversity. 


2.3. Data analysis 

The data transcription model at the focus group discussion (FGD) stage in this study is a verbatim 
transcript containing the name of the informant, profession, gender, interview time, and the content of the 
interview. Furthermore, the transcription process was carried out using Ms. Word and data encoding was 
created with NVivo 12 Plus software. The coding was produced with nodes in the NVivo software, where the 
code was adjusted to the theme or the UTAUT method, the PMT, and the DeLone and McLean information 
success model. The internal consistency was tested using Cronbach’s alpha scale, where a score of 0.7 or above 
was considered favorable [25]. An exploratory factor analysis with varimax rotation was carried out to 
investigate the domain structure [26]. The Kaisere Mayere Olkin (KMO) measure was used to determine the 
adequacy levels of a sample, where values greater than 0.5 indicates that the data were eligible for factor 
analysis. p-values of less than 0.05 were significant, and the IBM SPSS statistics 24 software was used to 
examine the data. 


3. RESULTS AND DISCUSSION 
3.1. Content validity 

The results of expert judgment involving five academics had a 100% response rate, and the formula 
produced values ranging from +1 to -1. Furthermore, the positive value showed that half of the panelists 
assessed the item as important/essential. The larger the distance of the Content Validity Index (CVT) from 0, 
the more “important” and higher the validity of its content. Expert validation results show that all items have 
an average content validity index value of more than 0.80. 


3.2. Face validity 

The data face validity from 10 respondents was collected in July 2022 through a web-based 
questionnaire using a google form. The data obtained were used to measure the simplicity, clarity, overlapping 
responses, balance, use of jargon, the accuracy of responses, use of technical language, application, and 
relationship to problems. Based on the assessment results, all the items met the face validity criteria. 


3.3. Final field study 

In this phase, information was collected with a web-based survey using a google form in July 2022. 
A total of 118 participants who were at least 18 years old filled the questionnaire. The participants were selected 
using convenience sampling from a variety of population groupings to achieve the greatest possible diversity. 
The description of the respondents characteristics are presented in Table 1. 

The questionnaires developed in this study were distributed to 118 respondents. Furthermore, 83.1% 
of them in the phase 2 pilot test were women, of which 60% were in the productive age and 50.8% had a 
Diploma degree. A total of 24% of the income was between the range of IDR 1,000,000-IDR 2,500,000. The 
majority of respondents worked in other sectors, namely 29.73%, while the application that is often used to 
find health information was HaloDoc, as stated by 58% of the population. 


3.4. Bias prevention 

The interpretation of study results can be negatively affected by several factors, including biased 
estimates of the validity and reliability, as well as inaccurate estimations of the correlations between 
components, which has an effect on the hypothesis testing. Several preventives, investigative, and remedial 
strategies that can be used to lessen worries regarding the potential effects of typical techniques were 
implemented to obtain the observed results. The most preferred method is the Harman Single Factor approach 
[27]. Table 2 shows the results of the Harman single factor technique, which estimated the variance of the 
general method, namely 35.85%. Furthermore, this was less than the maximum accepted threshold of 50% 
[27]. This indicates that the general method has no bias problem in the dataset of this study. 
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Table 1. Distribution of respondents’ characteristics in phase two of the trial based on age, gender, education, 
occupation, and income 


Characteristics Frequency Percentage (%) 
Age 
18-24 years 73 60 
25-44 years 41 35 
45-64 years 4 4 
Gender 
Male 20 16.9 
Female 98 83.1 
Education 
Senior high school 40 33.9 
Diploma 60 50.8 
Bachelor 7 5.9 
Master 10 8.5 
Doctoral 1 0.8 
Occupation 
Private employee 22 18.6 
Student 61 51.7 
Government officer 4 3.4 
Jobless 6 5.1 
Entrepreneur 25 21.2 
Income (IDR) 
Not yet earning 22 18.6 
<IDR 1,000,000 32 27.1 
IDR 1,000,000-IDR 2,500,000 24 20.3 
IDR 2,500,000-IDR 3,500,000 15 12.7 
>IDR 3,500,000-IDR 5,000,000 11 9.3 
>IDR 5,000,000-IDR 10,000,000 10 8.5 
>IDR 10,000,000 4 4 
Types of application 
Alodokter 20 16.9 
Pharmacies 1 0.8 
Grab health 2 1.7 
Halodoc 69 58.5 
K24 online 12 10.2 
Klikdokter 1 0.8 
Shopee 11 9.3 
Tokopedia 2 1.7 


Table 2. Result analysis common method bias 
Total Variance (%) Cumulative (%) 
15.77 35.85 35.85 


3.5. Construct validity 

Exploratory factor analysis (EFA) was carried out with the principal component approach and the 
varimax rotation method. The adequacy of the sample was evaluated using the Kaiser-Myer-Olkin (KMO) test 
and Bartlett's test of sphericity. The analysis results revealed that the KMO value was 0.868, which indicated 
that the data was adequate for the factor analysis [28]. The EFA of the main components with varimax rotation 
was used to build the structure and test the construct validity of an instrument. Factors with eigenvalues >1 or 
cumulative contribution rates >70% were retained and selected for interpretation [29]. In the first round, 
thirteen factors with eigenvalues greater than 1 were extracted, as shown in Figure 1. They can also explain 
69.88% of the total variance of the hospital outpatient service model. However, these items have a low loading 
factor value of <0.5, which is known as cross-loading due to several considerations. The items were also not 
arranged as expected, hence, the structure extracted in the first round was not acceptable. After the process, 5 
items were eliminated from the analysis, namely CSEF1, EFEX2, PEEX1, PEEX2, and PEEX3. The EFA was 
then repeated to obtain an acceptable structure. In the second round, factor extraction was carried out through 
principal component analysis and varimax rotation. This process produced 8 factors solution consisting of 
15.54% social influence and support; 11.06% understanding of proficiency, confidence and quality of 
information; 6.64% condition of facilities, skills; 7.87% severity and suffering; 7.74%-time opportunity 
support and system quality; 7.34% telepharmacy adoption intention; 6.80% perceived vulnerability; 6.37% 
attitude. All the constructs formed were able to explain 72.40% of the total variance, as shown in Figure 1. The 
outcome demonstrated the same factor structure as the initial scale for the purpose of validating our one-factor 
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reading difficulty are methodological effects that contribute to these associated errors [30]. 


The validity analysis using the principal component analysis approach produced 39 items with 
eigenvalues greater than |. The results of the principal component analysis were not equal to that of the original 
study. This was because it lacked the tools to match the findings of this current study. However, the quesionaire 
has only one single-factor because the key element assessment showed that every article in the screen-plot 


analysis has an initial component and an obvious pattern [31]. 
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Table 3 shows the results of the validity test using the principal component analysis approach on all 
variables. The loading factor value of all items was above 0.50 and each group of variables was in one 
component. Furthermore, all the question items were in their groups, which indicates their validity. 
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Figure 1. Scree plot EFA second round 


Table 3. Result of convergen validity test 


Factor Name of the construct Item Loading Factor Name of the construct Item Loading 
code factor code factor 
1 Efficacy response, social RESE1 0.665 5 Support, promised SRRQ2 0.527 
influence, and support RESE2 0.568 time, system quality SRRQ3 0.561 
RESE3 0.594 SYTQI 0.630 
RESE4 0.698 SYTQ2 0.794 
SOIN1 0.823 SYTQ3 0.686 
SOIN2 0.819 
SOIN3 0.742 
SOIN4 0.788 
SRRQI 0.556 
2 Understanding, proficient, CSEF2 0.623 6 Intention to adopt INHS1 0.835 
confident, and information CSEF3 0.578 telepharmacy INHS2 0.805 
quality EFEX1 0.666 INHS3 0.795 
INFQ1 0.601 
INFQ2 0.766 
INFQ3 0.715 
INFQ4 0.572 
3 Facilitating condition, easy EFEX3 0.626 7 Perceived vulnerability PEVU1 0.894 
skillful FACO1 0.682 PEVU2 0.875 
FACO2 0.800 PEVU3 0.893 
FACO3 0.750 
FACO4 0.708 
4 Severity and suffer PESE1 0.860 8 Attitude ATTIL 0.758 
PESE2 0.898 ATTI2 0.679 
PESE3 0.888 ATTIS 0.722 
PEVU4 0.591 
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3.6. Reliability test 

Table 4 shows that the lowest Cronbach Alpha value was 0.778, while the highest was 0.926, which 
indicates the reliability of all the variables. The overall Cronbach alpha of the variables in this study was 0.87. 
For factors 1 to 8, values of 0.926, 0.887, 0.852, 0.867, 0.880, 0.899, 0.892, and 0.778 were obtained, 
respectively. The calculation of the coefficient alpha provides information on the scale’s dependability. 
Therefore, Cronbach alpha is often used to assess the validity of study’s items. It is also a reliable coefficient 
that shows the relationships between elements in the collection that are proportionally associated to one 
another. Based on the assessment results, reliability score below 0.60 was regarded as weak [32]. 


Table 4. Result of reliability test 


Factor Name of the construct Cronbach alpha value Cut-off value 
1 Eficaccy response, social influence, and support 0.926 
2 Understanding, proficient, confident, and information quality 0.887 
3 Facilitating condition, easy skillful 0.852 
4 Severity and suffer 0.867 >0.70 
5 Support, promised time, and system quality 0.880 
6 Intention to adopt telepharmacy 0.899 
7 Perceived vulnerability 0.892 
8 Attitude 0.778 


Although efforts were made to cover a varied population, this study’s drawbacks include its inability 
to show predictive and concurrent validity, which requires a lengthy follow-up, as well as its inadequate 
representativeness of lower socioeconomic strata. Therefore, more studies are needed in several societal sectors 
and geographical areas. The eight constructs of telepharmacy service adoption were examined in this study 
using a 39-item scale. Confirmatory factor analysis must be carried out on the same scale with a bigger sample 
to further validate the study instrument. This measure needs to be improved and tested in other contexts to 
determine the generalizability. 


4. CONCLUSION 

A valid and reliable questionnaire was developed in this study, which can be used to assess the 
behavior of telepharmacy adoption during the COVID-19 pandemic crisis. It can also highlight the various 
factors associated with adoption behavior, thereby assisting pharmacists and government authorities to work 
towards successful digitization of pharmaceutical services by building telepharmacy literacy. 
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